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Abstract — There have been a number of studies on sparse sig- 
nal recovery from one-bit quantized measurements. Nevertheless, 
little attention has been paid to the choice of the quantization 
thresholds and its impact on the signal recovery performance. 
This paper examines the problem of one-bit quantizer design for 
sparse signal recovery. Our analysis shows that the magnitude 
ambiguity that ever plagues conventional one-bit compressed 
sensing methods can be resolved, and an arbitrarily small 
reconstruction error can be achieved by setting the quantization 
thresholds close enough to the original unquantized measure- 
ments. Note that the unquantized data are inaccessible by us. 
To overcome this difficulty, we propose an adaptive quantization 
method that iteratively refines the quantization thresholds based 
on previous estimate of the sparse signal. Numerical results are 
provided to collaborate our theoretical results and to illustrate 
the effectiveness of the proposed algorithm. 

Index Terms — One-bit compressed sensing, adaptive quantiza- 
tion, quantization design. 



I. Introduction 

Compressive sensing is a recently emerged paradigm of 
signal sampling and reconstruction, the main purpose of which 
is to recover sparse signals from much fewer linear measure- 
ments fl], |0 



y 



Ax 



(1) 



where A £ flj mx ™ j s the sampling matrix with m <C n, and x 
denotes the n-dimensional sparse signal with only K nonzero 
coefficients. Such a problem has been extensively studied 
and a variety of algorithms that provide consistent recovery 
performance guarantee were proposed, e.g. |fl]-G|. Conven- 
tional compressed sensing assumes infinite precision of the 
acquired measurements. In practice, however, signals need to 
be quantized before further processing, that is, the real-valued 
measurements need to be mapped to discrete values over some 
finite range. Besides, in some sensing systems (e.g. distributed 
sensor networks), data acquisition is expensive due to limited 
bandwidth and energy constraints [5 1. Aggressive quantization 
strategies which compress real-valued measurements into one 
or only a few bits of data are preferred in such scenarios. 
Another benefit brought by low-rate quantization is that it can 
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significantly reduce the hardware complexity and cost of the 
analog-to-digital converter (ADC). As pointed out in [6 1, low- 
rate quantizer can operate at a much higher sampling rate than 
high-resolution quantizer. This merit may lead to a new data 
acquisition paradigm which reconstructs the sparse signal from 
over-sampled but low-resolution measurements. 

Inspired by practical necessity and potential benefits, com- 
pressed sensing based on low-rate quantized measurements 
has attracted considerable attention recently, e.g. G]-||9]]. In 
particular, the problem of recovering a sparse or compressible 
signal from one-bit measurements 



b = sign(y) = sign(Acc) 



(2) 



was firstly introduced by Boufounos and Baraniuk in their 
work iflOl . where "sign" denotes an operator that performs 
the sign function element-wise on the vector, the sign function 
returns 1 for positive numbers and —1 otherwise. Following 
IflOl . the reconstruction performance from one-bit measure- 
ments was more thoroughly studied and a variety of one- 
bit compressed sensing algorithms such as binary iterative 
hard thresholding (BIHT) 0j], matching sign pursuit (MSP) 
[12], and many others [13|-[15| were proposed. Despite all 
these efforts, very little is known about the optimal choice 
of the quantization thresholds and its impact on the recovery 
performance. In most previous work (e.g [6|, [10|-[15|), the 
quantization threshold is set equal to zero. Nevertheless, such 
a choice does not necessarily lead to the best reconstruction 
accuracy. In fact, in this case, only the sign of the measurement 
is retained while the information about the magnitude of the 
signal is lost. Hence an exact reconstruction of the sparse 
signal is impossible without additional information regarding 
the sparse signal. In some other scenarios such as intensity- 
based source localization in sensor networks, since all sampled 
measurements are non-negative, comparing the original mea- 
surements with zero yields all ones wherever the sources are 
located, which makes identifying the true locations impossible. 
From the above discussions, we can see that quantization is 
an integral part of the sparse signal recovery and is critical to 
the recovery performance. 

There have been some interesting work [16|-[18| on quan- 
tizer design for compressed sensing. Specifically, the work 
1 16] utilizes the high-resolution distributed functional scalar 
quantization theory for quantizer design, where the quanti- 
zation error is modeled as random noise following a certain 
distribution. Nevertheless, such modeling holds valid only for 
high-resolution quantization, and may bring limited benefits 
when a low-rate quantization strategy is adopted. In lfT7l . lfl8l . 
authors proposed a generalized approximate message passing 



(GAMP) algorithm for quantized compressed sensing, and 
studied the quantizer design under the GAMP reconstruction. 
The method [ 17], 1 18 1 was developed in a Bayesian framework 
by modeling the sparse signal as a random variable and 
minimizing the average reconstruction error with respective 
to all possible realizations of the sparse signal. This method, 
however, involves high-dimensional integral or multidimen- 
sional search to find the optimal quantizer, which makes a 
close-form expression of the optimal quantizer design difficult 
to obtain. In |[T8l . an adaptive algorithm was proposed which 
adjusts the thresholds such that the hyperplanes pass through 
the center of the mass of the probability distribution of the esti- 
mated signal. Albeit intuitive, no rigorous theoretical guarantee 
was available to justify the proposed adaptive method. The 
problem of reconstructing image from one-bit quantized data 
is considered in |fl9l , where the quantization thresholds are 
set such that the binary measurements are acquired with equal 
probability. Such a scheme has been shown to be empirically 
effective. Nevertheless, it is still unclear why it leads to better 
reconstruction performance. 

In this paper, we investigate the problem of one-bit quan- 
tization design for recovery of sparse signals. The study is 
conducted in a deterministic framework by treating the sparse 
signal as a deterministic parameter. We provide an quantita- 
tive analysis which examines the choice of the quantization 
thresholds and quantifies its impact on the reconstruction 
performance. Analyses also reveal that the reconstruction error 
can be made arbitrarily small by setting the quantization 
thresholds close enough to the original unquantized mea- 
surements y. Since the original unquantized data samples y 
are inaccessible by us, to address this issue, we propose an 
adaptive quantization approach which iteratively adjusts the 
quantization thresholds such that the thresholds eventually 
come close to the original data samples and thus a small 
reconstruction error is achieved. 

The rest of the paper is organized as follows. In Section 
HTl we introduce the one-bit compressed sensing problem. 
The main results of this paper are presented in Section [jlj] 
where we show that, by properly selecting the quantization 
thresholds, sparse signals can be recovered from one-bit mea- 
surements with an arbitrarily small error. A rigorous proof 
of our main result is provided in Section [TV] An adaptive 
quantization method is developed in [V] to iteratively refine 
the thresholds based on previous estimate. In Section [VI] 
numerical results are presented to corroborate our theoreti- 
cal analysis and illustrate the effectiveness of the proposed 
algorithm, followed by concluding remarks in Section IVIII 

II. Problem Formulation 

We consider a coarse quantization-based signal acquisition 
model in which each real-valued sample is encoded into one- 
bit of information 



b = sign(y — r) = sign(Acc — t) 



(3) 



where b — [b\ 62 ■ • • b m ] T are the binary observations, and 
t = [t\ ti ... T m ] T denotes the quantization threshold vec- 
tor. As mentioned earlier, in most previous one -bit compressed 



sensing studies, the quantization thresholds are set equal to 
zero, i.e. r = 0. A major issue with the zero quantization 
threshold is that it introduces a magnitude ambiguity which 
cannot be resolved without additional information regarding 
the sparse signal. The reason is that when r = 0, multiplying 
x by any arbitrary nonzero scaling factor will result in the 
same quantized data {b n }. To circumvent the magnitude ambi- 
guity issue, in ifTD . [13], a unit-norm constraint is imposed on 
the sparse signal and it was shown that unit-norm signals can 
be recovered with a bounded error from one bit quantized data. 
Nevertheless, the choice of the zero threshold is not necessarily 
the best. In this paper, we are interested in examining the 
choice of the quantization thresholds and its impact on the 
reconstruction performance. 

Specifically, we consider the following canonical form for 
sparse signal reconstruction 



min||;z||o s.t. sign(A;z 



(4) 



Such an optimization problem, albeit non-convex, is more 
amenable for theoretical analysis than its convex counterpart 
( HTt . Suppose x is a solution of the above optimization. In the 
following, we examine the choice of r and its impact on the 
reconstruction error \\x — x\\2- 

III. One-Bit Quantization Design: Analysis 

To facilitate our analysis, we decompose the quantization 
threshold vector r into a sum of two terms: 

r = y 8 (5) 

where 8 — [61 62 ... S m ] T can be treated as a constrained 
deviation from the original unquantized measurements y. 
Substituting (0 into 01, we get 

b = sign(Aa; — t) = sign(£) (6) 

Suppose x is the solution of (@), Let h = x — x be the 
residual (reconstruction error) vector. Clearly, h is a 2i^-sparse 
vector which has at most IK nonzero entries since x has 
at most K nonzero coefficients. Also, the solution x yields 
estimated measurements that are consistent with the observed 
binary data, i.e. 

b = sign(A:r — r) = sign(Ah + 8) (7) 

Combining (0 and (0, the residual vector h has to satisfy the 
following constraint 



sign(Ah + 8) = sign((5) 



(8) 



In the following, we will show that the residual vector h 
can be bounded by the deviation vector 8 if the number 
of measurements are sufficiently large and A satisfies a 
certain condition. Thus the reconstruction error h can be made 
arbitrarily small by setting 8 close to zero. Our main results 
are summarized as follows. 

Theorem 1: Let x e W 1 be an X-sparse vector. A G M mx ™ 
is the sampling matrix. Suppose there exist an integer k > 2K 
and a positive parameter ji such that any k x n submatrix A 
constructed by selecting certain rows of A satisfies 



\Au\\ 2 2 > fi\\u\\ 



(9) 



for all 2A-sparse vector u. Also, assume that each entry of 
8 is independently generated from a certain distribution with 
equal probabilities being positive or negative. Let x denote the 
solution of the optimization problem |@). For any arbitrarily 
small value r\ > 0, we can ensure that the following statement 
is true with probability exceeding 1 — -q: if the number of 
measurements, m, is sufficiently large and satisfies 

m — IK log(m — k + 1) — (k — 1) log m 
> 2K(\og(ne 2 ) - 2 \og(2K) + 1) + log(l/??) + c (10) 

then the sparse signal can be recovered from © with the 
reconstruction error bounded by 



\x - x\\ 2 < —= = Ae 



(11) 



where e = ||<5|| 2 , A = 1/ ' JJl, e in ( fTOb represents the base of 
the natural logarithm, and c in ( fTOb is defined as 



c4( /s _l)(lo g (e/(/ S -l)) + l) 



(12) 



which is a constant only dependent on k. 

We have the following remarks regarding Theorem [TJ 
Remark 1: Clearly, the condition (|9]i is guaranteed if there 

exists a constant 72K < 1 such that any k x n submatrix of 

A satisfies 



(l + 7 2K )||u|||>||Au|||>(l 



72K) 



(13) 



for all 2A-sparse vector u. As indicated in previous studies 
|TJJ, when the sampling matrix is randomly generated from 
a Gaussian or Bernoulli distribution, this restricted isometry 
property is met with an overwhelming probability, particularly 
when k is large. Hence it is not difficult to find an integer k 
and a positive /1 such that the condition (0 holds. 

Remark 2: Note that the term on the left hand side of ([Tol is 
a monotonically increasing function of m when m is greater 
than a certain value. Thus for fixed values of K, k, n, and 
?/, we can always find a sufficiently large m to ensure the 
condition ( fTOb is guaranteed. In addition, a close inspection of 
( [Tol l reveals that, for a constant 77, the number of measurements 
required to guarantee ([Tot is of order 0(Klog(n)), which is 
the same as that for conventional compressed sensing. 

Remark 3: From (fTTT i, we see that the sparse signal can be 
recovered with an arbitrarily small error by letting S > 0, or 
in other words, by setting the threshold vector r sufficiently 
close to the unquantized data y. This result has two important 
implications. Firstly, sparse signals can be reconstructed with 
negligible errors even from one-bit measurements. We note 
that [TTJ, |[T3l also showed that sparse signals can be recovered 
with a bounded error from one bit quantized data. But then- 
analysis ignores the magnitude ambiguity and confines sparse 
signals on the unit Euclidean sphere. Such a sphere constraint 
is no longer necessary for our analysis. Secondly, our result 
suggests that the best thresholds should be set as close to 
the original data samples y as possible. So far there has no 
theoretical guarantee indicating that other choice of thresholds 
can also lead to a stable recovery with an arbitrarily small 
reconstruction error. 



IV. Proof of TheoremQ] 

Let af denote the zth row of the sampling matrix A. 
Clearly, to ensure the sign consistency dHJ for each component, 
we should either have 



or 



sign(afh) = sign(^) 



\afh\ < \6i 



(14) 



(15) 



Note that in the latter case the sign of (a[h + 5i) does not 
flip regardless of the sign of afh. Therefore if there exists a 
2A"-sparse residual vector h such that each component of Ah 
satisfies either (TBI or (Q3), then x = x + h is the solution 
of (01). Our objective in the following is to show that such a 
residual vector h is bounded by the deviation vector S with 
an arbitrarily high probability. 

Without loss of generality, we decompose A and S into two 
parts: A = [A^ A^p, S = [S^ S^] 7 ', according to the two 
possible relationships between af h and Sf. 



sign(Aih) = signal) 
\A 2 h\ < \S 2 \ 



(16) 



where A x e M miX ™, A 2 £ M m2Xn , m x + m 2 = m, and in 
the second equation, both the absolute value operation | • | and 
the inequality symbol "<" applies entrywise to vectors. 

We now analyze the probability of the residual vector h 
being greater than Ae, given that the condition ( TToT l is satisfied. 
This conditional probability can be denoted as P(||h||2 > 
Ae|E), where we use "E" to denote the event ( TToT l. Clearly, 
■P(||' l l|2 > Ae|E) can also be explained as the probability 
that the event \\h\\ 2 > Ae will occur, when the event E has 
occurred. To facilitate our analysis, we divide the event (IToT ) 
into two disjoint sub-events which are defined as 

El: The event ( TToT ) holds true for m 2 > k 
E2: The event ( TToT ) holds true for m 2 < k 

Clearly, the union of these two sub-events is equal to the event 
( IToT ). Utilizing Bayes' Theorem, the probability P(||h||2 > 
Ae|E) can be expressed as 

P(||/i||2>Ae|E) ( = ) P(||/i|| 2 >Ae,E) 

=P{\\h\\ 2 > Ae,ElUE2) 

( = ) P(||h|| 2 > Ae,El) + P(||fc|| 2 >Ae,E2) 

(17) 

where (a) holds because the events ( TToT l is a prerequisite 
condition that is always met to ensure the sign consistency, i.e. 
P(E) = 1, and (6) follows from the fact that the probability 
of the union of two disjoint events is equal to the sum of their 
respective probabilities. 

Let us first examine the probability P(||/i||2 > Ae,El). We 
show that the events \\h\\ 2 > Ae and El are two mutually 
exclusive events which cannot occur at the same time. To see 
this, note that when the event El occurs, we should have 



\A 2 h\ < \So 



\A 2 h 



I 2 < e 2 



(18) 



in which A 2 G M" l2Xn and m 2 > K. On the other hand, 
recalling that any k x n sub-matrix formed by selecting certain 
rows of A satisfies the condition (O, we have 

\\A 2 hf 2 > vi\\h\\l (19) 

Combining (fl~8T > and (T% , we arrive at 

||7»|| 2 <-^ = Ae (20) 

which is contradictory to the event \\h\\ 2 > Ae. Hence the 
events El and ||/i||2 > Ae cannot occur simultaneously, which 
implies 



P(||h|| 2 >Ae,El) = 



(21) 



Substituting (0) into (EJ, the probability P(\\h\\ 2 > Ae|E) 
is simplified as 



P(||/i|| a > Ae|E)=P(||/i|| 2 >Ae,E2) 



(22) 



The probability P(||/i||2 > Ae,E2), however, is still difficult 
to analyze. To circumvent this difficulty, we, instead, derive 
an upper bound on P(||h||2 > Ae,E2): 

P(||fr|| 2 >Ae,E2) 
=P(||/i||2 > Ae|E2)P(E2) < P(E2) 

= P(sign(Aih) = sign(di), \A 2 h\ < \S 2 \,m 2 < k) 
<P(sign(Aih) = sign(^i),m 2 < k) 
=P(sign(Ai/i) = sign(^i), mi > m — k) 

^(Ua) (23) 

^ t=0 ' 

where (a) comes from the definition of the event E2, and in 
(b), the event f2j is defined as 



Qi : sign(Ai^) = sign(<5i) 

where A 1 GM miX ", and mi 



(24) 



which means that there exist at least m — i components in 
Ah whose signs are consistent with the corresponding entries 
in 6. Note that since the rest i components are not explicitly 
specified in fij, the event Hi include all possibilities for the 
rest i components. Based on the definition of Oj, we can infer 
the following relationship: fl^ D tti 2 for i\ > i 2 . This is 
because tti 2 can be regarded as a special case of the event 
Q,^ with some of the unspecified i\ components also meeting 
the sign consistency requirement. With this relation, the upper 
bound derived in ( 1231 can be simplified as 



P(||fc|| 2 > Ae,E2) < P( (J Oil =P(0 K _i 



(25) 



We note that for the event K _i, A\ is not specified and can 
be any submatrix of A. Considering selection of m — n + 1 
rows (out of m rows of A) to construct A\, the event K _i 
can be expressed as a union of a set of sub-events 

J 

a,-! - |J fi£_ x (26) 



where J = C(m, m — K + 1), C(m, k) denotes the number 
of k combinations from a given set of m-elements, and each 
sub-event 0^_i is defined as 



ni 



sign(Aih) = sign(^i) where A\ = A[Ij, :} (27) 



in which Ij is an unique index set which consists of m — k + 
1 non-identical indices selected from {l,2,...,m}, A[ij,:] 
denotes a submatrix of A constructed by certain rows from 
A, and the indices of the selected rows are specified by Ij. 
From d26i>. we have 



p(o re _o=p U -i ^E p (°-i 



i=i 



(a) 



(28) 



j=i 



where the inequality (a) follows from the fact that the 
probability of a union of events is no greater than the sum 
of probabilities of respective events. The inequality becomes 
an equality if the events are disjoint. Nevertheless, the sub- 
events {£l J K _i} are not necessarily disjoint and may have 
overlappings due to the k — 1 unspecified components. 

We now analyze the probability P(J7^ C _ 1 ). To begin with 
our analysis, we introduce the concept of orthant originally 
proposed in ifTTI for analysis of one -bit compressed sensing. 
An orthant in E m is a set of vectors that share the same sign 
pattern, i.e. 



Oi = {)i£ R m |sign(«) = u] 



(29) 



A useful result concerning intersections of orthants by sub- 
spaces is summarized as follows. 

Lemma 1: Let S be an arbitrary fc-dimensional subspace 
in an m-dimensional space. Then the number of orthants 
intersected by S can be upper bounded by 



I(m,k) <2 k C(m,k) 



(30) 



where C(m, k) denotes the number of fc-combinations from a 
set of n-elements. 

Proof: See (2DJ Lemma 8]. ■ 

The probability P(flJ._ 1 ) of our interest can be interpreted 
as, the probability of the vector A\h lying in the same orthant 
as #i for a given A\. We first examine the number of sign 
patterns the vector A\h could possibly have. Let S denote the 
set of all possible sign patterns for A\h, i.e. 

S = {u = sign(Aih)\h G R™ is a 2iT-sparse vector, 

Ai = A[/ is :]} (31) 

Also, let T denote the support of the sparse residual vector h, 
we can write 



g ^ A 1 h = A 1 [:,T]h T 



(32) 



where g G R m_K+1 , and Ai[:,T] denotes a submatrix of A\ 
obtained by concatenating columns whose indices are specified 
by T. We see that g is a linear combination of 2K columns of 
A\, and thus g lies in an 2if-dimensional subspace spanned 
by the columns of A\\\, T], Recalling LemmaQ] we know that 
the number of orthants intersected by this subspace is upper 
bounded by 2 2K C(m — k + 1,2K). Therefore the vector g 
which lies in this subspace has at most 2 2K C(m— k+1, 2K) 



possible sign patterns. Note that this result is for a specific 
choice of the index set T. The selection of the support T from 
n entries has at most C(n, 2K) combinations. Therefore, in 
summary, the number of sign patterns in the set S is upper 
bounded by 



N SP < 2 ZK C(m - k + 1, 2K)C(n, 2K) 
The probability P(0^,_ 1 ) can be calculated as 
P(^_i) =P(sign(<5 1 ) G S) 



(33) 



(a) N< 



SP 



o(m— k+1) 



< 



2 2K C(m - k + 1, 2K)C(n, 2K) 



0(m— k+1) 



(34) 



where (a) comes from the fact that (5 is a vector whose entries 
are independently generated from a certain distribution with 
equal probabilities being positive and negative, and di has 
2(ra-K+i) p 0SS ible sign patterns. 

Combining (|22]), (|25]l, (|28]l and (O, we arrive at 



P(||h|| 2 > Ae|E) = P(||h|| 2 > Ae|E2) < P(0«_i) 
2 2K C(m -k+ l,2K)C(n, 2K)C(m, k - 1) 
— 2('™- K + 1 ) 

where the last inequality comes from J = C(to, to — n + 1) = 
C(to, k — 1). Utilizing the following inequality ifTTI 

C{a,b)<(^) h (36) 

in which e w 2.718 denotes the base of the natural logarithm, 
the probability P(||/i|| 2 > Ae|E) can be further bounded by 



where 



P{\\h\\ 2 > Ae|E) < | 



n(m — k + \)e* 



(37) 



(k-1) 



(2K) 2 ) \k-\ 

i A n(m—K+l—2K) 

Examine the condition which ensures that a/b is less than 
a specified value r\, where < r\ < 1. Taking the base-2 
logarithm on both sides of (a/b) < r\ and rearranging the 
equation, we obtain 

to — 2K log(m — k + 1) — (k — 1) log m 
> 2if (log(ne 2 ) - 2 log(2iT) + 1) + log(l/r/) + c (38) 

where 

c^( K -l)(log(e/( K -l)) + l) 

is a constant only dependent on k. In summary, for a specified 
r], if the condition (l38l is satisfied, then we can ensure that 
the probability of the residual vector h being greater than Ae 
is smaller than 77, i.e. 



P(||fc|| 2 >Ae|E)<77 



or 



P(N| 2 <Ae|E)>l-7? 
The proof is completed here. 



(39) 



(40) 



V. Quantization Design: Adaptive Methods 

In this section, we aim to develop a practical algorithm 
for one-bit compressed sensing. Previous analyses show that 
a reliable and accurate recovery of sparse signals is possible 
even based on one-bit measurements. This theoretical result is 
encouraging but we are still confronted with two practical diffi- 
culties while trying to recover the sparse signal via solving (0J. 
Firstly, the optimization (@]i is a non-convex and NP hard prob- 
lem that has computational complexity growing exponentially 
with the signal dimension n. Hence alternative optimization 
strategies which are more computationally efficient in finding 
the sparse solution are desirable. Secondly, our theoretical 
analysis suggests that the quantization thresholds should be set 
as close as possible to the original unquantized measurements 
y. However, in practice, the decoder does not have access 
to the unquantzed data y. To overcome this difficulty, we 
will consider a data-dependent adaptive quantization approach 
whereby the quantization threshold vector is dynamically 
adjusted from one iteration to next, in a way such that the 
threshold come close to the desired values. 

A. Computational Issue 

To circumvent the computational issue of (0), we can 
replace the ^o- norm with alternative sparsity-promoting func- 
tional. The most popular alternative is the i?i-norm. Replacing 
the ^o- norm with this sparsity-encouraging functional leads to 
the following optimization 



|z||i s.t. sign(A,z 



(41) 



which is convex and can be recast as a linear programming 
problem that can be solved efficiently. Although a rigorous 
theoretical justification for l\ -minimization based optimization 
is still unavailablqj, our simulation results indeed suggest that 
(PiTT i is an effective alternative to Iq -minimization and is able to 
yield a reliable and accurate reconstruction of sparse signals. 
In addition to the i!i-norm, another alternative sparse- 
promoting functional is the log-sum penalty function. The 
optimization based on the log-sum penalty function can be 
formulated as 



min > log( 
2 < * 



e) s.t. sign(Az — r) = b 



(42) 



where e > is a parameter ensuring that the penalty func- 
tion is well-defined. Log-sum penalty function was originally 
introduced in [21 1 for basis selection and has gained increas- 
ing attention recently El . Il23l . Experiments and theoretical 
analyses show that log-sum penalty function behaves more 
like £o-norm than ^i-norm, and has the potential to present 
superiority over the ^i-minimization based methods. The op- 
timization (l42l can be efficiently solved by resorting to a 
bound optimization technique [22|-[24|. The basic idea is to 
construct a surrogate function Q(z\z^') such that 



Q(z\z^)-L(z)>0 



(43) 



'A theoretical guarantee for £1 -minimization is under study and will be 
provided in our future work. 




Fig. 1. Schematic of one-bit adaptive quantization for compressed sensing. 



where L(z) is the objective function and the minimum is 
attained when z 



l {t \ i.e. Q(z (t) \z {t) ) = L{z {t) ). Op- 
timizing L(z) can be replaced by minimizing the surrogate 
function Q[z\zS') iteratively. An appropriate choice of such 
a surrogate function for the objective function (l42l is given 
by 



Q(z\2®) = 



i=\ 



,Wi 



+ e 



+ log(|z 4 (t) |+ e )- 




Therefore optimizing (l42l can be formulated as reweighted 
^i-minimization which iteratively minimizes the following 
weighted l\ function: 



min Q(z\z {t) ) = V 

z *- — J 

sign(A;z — t) - 



(*)i 

W) '\Zi 



+ constant 



s.t. 



(45) 



.,(*) 



where the weighting parameters are given by w} 
l/(\ z i I + e), Vi. The optimization (l45l l is a weighted 
version of (14Tb and can also be recast as a linear programming 
problem. By iteratively minimizing (RTt . we can guarantee that 
the objective function value L(x) is non-increasing at each 
iteration. In this manner, the reweighted iterative algorithm 
eventually converges to a local minimum of ( l42l i. 

B. Adaptive Quantization 

As indicated earlier, in addition to the computational issue, 
the other difficulty we face in developing a practical algo- 
rithm is that the suggested optimal quantization thresholds are 
dependent on the original unquantized data samples y which 
are inaccessible to the decoder. To overcome this difficulty, 
we consider an adaptive quantization strategy in which the 
thresholds are iteratively refined based on the previous esti- 
mate/reconstruction. 

The basic idea of one-bit adaptive quantization is described 
as follows. At iteration t, we compute the estimated measure- 
ments y^' at the decoder based on the sparse signal x^ ~ ' 
recovered in the previous iteration: y"> = AxS ~ '. This 
estimate is then used to update the quantization thresholds: 
T (<) = y\> _|_ gW f where <r*' is a vector randomly generated 
from a certain distribution. In practice, the deviation 6"' 
should be gradually decreased to ensure that the thresholds 
will eventually come close to y. The updated thresholds r^ 
are then fed back to the encoder. At the encoder, we compare 
the unquantized measurements y with the updated thresholds 
T^\ and obtain a new set of one-bit measurements b"', 
which are sent to the decoder. Based on t^' and the new 



data 6^, at the decoder, we compute a new estimate of the 
sparse signal x' ' via solving the optimization fiTT i or d42l >. 
A schematic of the proposed adaptive quantization scheme is 
shown in Fig. [8] Note that throughout this iterative process, 
the unquantized measurements y are unchanged. For clarity, 
the one-bit adaptive quantization scheme is summarized as 
follows. 

One-bit adaptive quantization scheme 

1) Given an initial estimate y , and randomly generate 
an initial deviation vector S^ ' according to a certain 
distribution. 

2) At iteration t > 0, let y (t) = y (0) if t = 0; otherwise 



compute iy- 1 ' as y^' 



Ax {t ' 



Based on y^\ update 
the thresholds as: r (t) = y (t) + <S (t) , where £ (t) for 
t > is randomly generated according to a certain 
distribution with a decreasing variance. Compare y with 
the updated thresholds T^' and obtain a new set of one- 
bit measurements b^' . 
3) Based on r^ and &' ', compute a new estimate of the 
sparse signal a;' ' via solving the optimization ( f4TT > or 



4) Go to Step 2 if \\x 



*« 



x^\ 



> l), where w is a 



prescribed tolerance value; otherwise stop. 

As indicated in |6), Qjp , an important benefit brought by 
the one-bit design is the significant reduction of the hardware 
complexity. One-bit quantizer which takes the form of a simple 
comparator is particularly appealing in hardware implemen- 
tations, and can operate at a much higher sampling rate 
than the high-resolution quantizer. Also, one-bit measurements 
are much more amiable for large-scale parallel processing 
than high-resolution data. With these merits, the proposed 
adaptive architecture enables us to develop data acquisition 
devices with lower-cost and faster speed, meanwhile achieving 
reconstruction performance similar to that of using multiple- 
bit quantizer. 

The proposed adaptive quantization scheme shares a sim- 
ilar architecture with the method |[T8l in that both methods 
iteratively refine the thresholds using estimates obtained in 
previous iteration. Nevertheless, the rationale behind these 
two methods are different. Our proposed adaptive quantization 
scheme is based on Theorem Q] which suggests that an arbitrar- 
ily small reconstruction error can be attained if the thresholds 
are set close enough to the unquantized measurements y, 
whereas for [fljO, a similar theoretical guarantee is unavailable. 

VI. Numerical Results 

We now carry out experiments to corroborate our previous 
analysis and to illustrate the performance of the proposed 
adaptive quantization scheme. In our simulations, the if-sparse 
signal is randomly generated with the support set of the sparse 
signal randomly chosen according to a uniform distribution. 
The signals on the support set are independent and identically 
distributed (i.i.d.) Gaussian random variables with zero mean 
and unit variance. The measurement matrix A E R mx ™ j s 
randomly generated with each entry independently drawn from 
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Fig. 2. Reconstruction mean-squared error versus the number of measurements for different choices of a. 



Gaussian distribution with zero mean and unit variance. To 
circumvent the difficulty of solving (|4), we replace the £q norm 
with alternative sparsity-encouraging functionals, namely, t\ 
norm and the log-sum penalty function. The new formulated 
optimization problems (c.f. (|4TT > and (|42| |) can be efficiently 
solved. 

A. Performance under Different Threshold Choices 

We first examine the impact of the quantization design on 
the reconstruction performance. The knowledge of the original 
unquantized measurements y is assumed available in order to 
validate our theoretical results. The thresholds are chosen to be 
the sum of the unquantized measurements y and a deviation 
term S, i.e. r = y+S, where S is a vector with its entries being 
independent discrete random variables with P{5i = —a) = 0.5 
and P(8i = a) = 0.5, in which the parameter a > controls 
the deviation of r from y. Fig. |2] depicts the reconstruction 
normalized mean squared error (NMSE), E[ |j~jp ], vs. the 
number of measurements m for different choices of a, where 
we set n — 50, and K = 3. Results are averaged over 10 4 
independent runs. From Fig. [2] we see that the reconstruction 
performance can be significantly improved by reducing the 
deviation parameter a. In particular, a NMSE as small as 
10 -6 can be achieved when a is set 0.001. This corroborates 
our theoretical analysis that sparse signals can be recovered 
with an arbitrarily small error by letting 6 —t 0. Also, as 
expected, the reconstruction error decreases with an increasing 
number of measurements m. Nevertheless, the performance 
improvement due to an increasing m is mild when m is large. 
This fact suggests that the choice of quantization thresholds 
is a more critical factor than the number of measurements in 
achieving an accurate reconstruction. From Fig. |2] we also 
see that t\ -minimization and log-sum minimization provide 
similar reconstruction performance. In Fig. |3] we plot the root 
mean squared error (RMSE), E[\\x — &H2], as a function of 
the deviation magnitude e = ||<5||2 = s/ma, where we set 
m = 100, n = 120, K = 2 and a varies from 10~ 3 to 1. It 



can be observed that the RMSE decreases proportionally with 
the value e, which coincides with our theoretical analysis (fTTT i. 

To further corroborate our analysis, we consider a differ- 
ent way to generate the deviation vector d, with its entries 
randomly generated according to a Gaussian distribution with 
zero mean and variance a 2 . Fig. |4] depicts the NMSE vs. the 
number of measurements m for different values of a. Again, 
we observe that a more accurate estimate is achieved when 
the thresholds get closer to the unquantized measurements y. 

Experiments are also carried out on real world images in 
order to validate our theoretical results. As it is well-known, 
images have sparse structures in certain over-complete basis, 
such as wavelet or discrete cosine transform (DCT) basis. In 
our experiments, we sample each column of the 256 x 256 
image using a randomly generated measurement matrix A £ 
I^mx256. We then quantize each real-valued measurement into 
one bit of information by using the threshold r, — Hi + Si, in 
which Si is drawn from a Gaussian distribution Af(0, 0.01). 
Fig. [5] shows the original image and the reconstructed images 
based on m x 256 one-bit measurements, where m is equal 
to 150, 200, and 250, respectively. We see that the images 
restored from one-bit measurements still provide a decent 
effect, given that the thresholds are well-designed. 

B. Performance of Adaptive Quantization Scheme 

We now carry out experiments to illustrate the perfor- 
mance of the proposed adaptive quantization algorithm. For 
simplicity, the algorithm uses the optimization (|4TT > at Step 
3 of each iteration. In our experiments, we set n — 50, 
K - - 2, t(°) — 5' ' and the threshold vector is updated 



as t 



(*) 



= y {t) + £(*)<5 (t) for t > 0, where <5 (t) ,W is a 
random vector with its entries following a normal distribution 
with zero mean and unit variance. The parameter £w controls 
the magnitude of the deviation error. We set £(°> = 1, and 
to gradually decrease the deviation error, £w is updated 
according to £ (t+1) = £ (t) / 10 - The NMSE vs. the number 
of iterations is plotted in Fig. [6] where we set m = 40. 



10" 



10" 



|io 2 

DC 



I I I I I I 1 1 1 I I I I I I 1 1 1 — 1 1 — I I I I 1 1 1 

-i — Log-sum minimization 
'*■" L1 minimization 



10" 

10 J 10"' 10"' 

Deviation(s) 

Fig. 3. Root mean-squared error versus e. 



10 u 



10' 




5 10 15 

Number of iterations 

Fig. 6. Normalized mean-squared error versus number of iterations. 



20 



Results are averaged over 10 3 independent runs, with the 
sampling matrix and the sparse signal randomly generated 
for each run. From Fig. [6] we see that the adaptive algo- 
rithm provides a consistent performance improvement through 
iteratively refining the quantization thresholds, and usually 
provides a reasonable reconstruction performance within only 
a few iterations. Fig. [7] depicts the NMSEs as a function of the 
number of measurements for the one-bit adaptive quantization 
scheme and a non-adaptive one-bit scheme which uses t^ ' as 
its thresholds. For the adaptive scheme, the iterative process 
stops if ||ar'' — x" ^||a < 0.01. Numerical results show that 
the adaptive scheme usually converges within ten iterations. 
We observe from Fig. [7] that the adaptive scheme presents a 
clear performance advantage over the non-adaptive method. 

We examine the effectiveness of the adaptive quantization 
scheme for image recovery. In the experiments, we sample 
each column of the 128 x 128 image using a randomly gen- 
erated measurement matrix A £ ]p>"ixi28^ an( j tnen quantize 
each real-valued measurement into one bit of information. The 
initial threshold vector is set to be t^ — y + £^5' ', where 
£(°) is chosen as ||y||j/m such that the deviation is comparable 
to the magnitude of entries in y. For t > 0, the threshold is 
then updated as t® = y {t) + ^ S (t) , with fW = f^-^/lO. 
Fig. |B] plots the images reconstructed by the proposed one-bit 
adaptive scheme and the non-adaptive one-bit scheme which 
uses t^ as its thresholds, where we set m = 256. Fig. [8] 
demonstrates that the adaptive quantization scheme improves 
the reconstruction of the image significantly. 

VII. Conclusion 

We studied the problem of one-bit quantization design for 
compressed sensing. Specifically, the following two questions 
were addressed: how to choose quantization thresholds, and 
how close can the reconstructed signal be to the original 
signal when the quantization thresholds are well-designed? 
Our analysis revealed that sparse signals can be recovered with 
an arbitrarily small error by setting the thresholds close enough 
to the original unquantized measurements. The unquantized 
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Fig. 7. Normalized mean-squared errors versus the number of measurements. 





(a) 



(b) 



Fig. 8. Reconstruction of images from one-bit measurements: (a) Image 
reconstructed by non-adaptive one-bit scheme; (b) Image reconstructed by 
adaptive one-bit scheme. 



measurements, unfortunately, are inaccessible to us. To address 
this issue, we proposed an adaptive quantization method which 
iteratively refines the quantization thresholds based on previ- 
ous estimate. Simulation results were provided to corroborate 
our theoretical analysis and to illustrate the effectiveness of 
the proposed adaptive quantization scheme. 
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Fig. 4. Reconstruction mean-squared error versus the number of measurements for different choices of a. 







Fig. 5. Original image and reconstructed images based on m X 256 one-bit measurements. 
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